Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance plugin logging to provide more context information and request IDs #9853

Open
wants to merge 23 commits into
base: develop
Choose a base branch
from

Conversation

dmvrtx
Copy link
Contributor

@dmvrtx dmvrtx commented Dec 2, 2024

Fixes #9892

Changes proposed in this Pull Request

  • Include unique request ID, generated upon plugin initialization, into each log entry.
  • Include sequential entry numbers into the log.
  • Use JSON to format structured data in API client.
  • Ability to provide additional context for the request entry, e.g. order ID, payment intention ID, which can be used later for parsing.
  • Improve formatting of multi-line log entries to have the same prefix as the first line for better readability and processing by tools like grc.

Testing instructions

  • Automated tests shall pass.
  • Checkout branch and visit admin pages of the plugin. Check the woopayments-YYYY-MM-DD... logs in the wp-content/uploads/wc-logs/ and see that the formatting is in place, e.g. like below:
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0004 REQUEST GET https://public-api.wordpress.com/wpcom/v2/sites/%s/wcpay/disputes?test_mode=1&pagesize=50&search%5B0%5D=warning_needs_response&search%5B1%5D=needs_response
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0004 CONTEXT: {
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0004     "WP_User": "Guest (non logged-in user)",
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0004     "HTTP_REFERER": "http:\/\/wcpay-debian.vm.local:8082\/wp-admin\/admin.php?page=wc-admin&path=paymentstransactions",
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0004     "HTTP_USER_AGENT": "Mozilla\/5.0 (Macintosh; Intel Mac OS X 10.15; rv:133.0) Gecko\/20100101 Firefox\/133.0",
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0004     "REQUEST_URI": "\/wp-admin\/admin.php?page=wc-admin&path=paymentstransactionsdetails&id=pi_3QRxAsR5iZ61kaPg13Bz8Ph6&transaction_id=txn_3QRxAsR5iZ61kaPg1mZ5lQYe&transaction_type=charge",
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0004     "DOING_AJAX": "",
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0004     "DOING_CRON": "",
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0004     "WP_CLI": ""
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0004 }
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0005 HEADERS: {
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0005     "Content-Type": "application\/json; charset=utf-8",
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0005     "User-Agent": "WooCommerce Payments\/8.5.1"
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0005 }
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0006 RESPONSE: {
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0006     "data": []
2024-12-05T08:38:43+00:00 INFO 6751668fafe68-0006 }

or with grc running:

image
  • Run npm run changelog to add a changelog file, choose patch to leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.
  • Covered with tests (or have a good reason not to test in description ☝️)
  • Tested on mobile (or does not apply)

Post merge

@botwoo
Copy link
Collaborator

botwoo commented Dec 2, 2024

Test the build

Option 1. Jetpack Beta

  • Install and activate Jetpack Beta.
  • Use this build by searching for PR number 9853 or branch name dev/hack-logging-enhancements in your-test.site/wp-admin/admin.php?page=jetpack-beta&plugin=woocommerce-payments

Option 2. Jurassic Ninja - available for logged-in A12s

🚀 Launch a JN site with this branch 🚀

ℹ️ Install this Tampermonkey script to get more options.


Build info:

  • Latest commit: 269930d
  • Build time: 2024-12-16 14:31:18 UTC

Note: the build is updated when a new commit is pushed to this PR.

Copy link
Contributor

github-actions bot commented Dec 2, 2024

Size Change: 0 B

Total Size: 1.39 MB

ℹ️ View Unchanged
Filename Size
release/woocommerce-payments/assets/css/admin.css 1.37 kB
release/woocommerce-payments/assets/css/admin.rtl.css 1.37 kB
release/woocommerce-payments/assets/css/success.css 182 B
release/woocommerce-payments/assets/css/success.rtl.css 184 B
release/woocommerce-payments/dist/blocks-checkout-rtl.css 2.63 kB
release/woocommerce-payments/dist/blocks-checkout.css 2.63 kB
release/woocommerce-payments/dist/blocks-checkout.js 54.9 kB
release/woocommerce-payments/dist/cart-block.js 17 kB
release/woocommerce-payments/dist/cart.js 5.73 kB
release/woocommerce-payments/dist/checkout-rtl.css 932 B
release/woocommerce-payments/dist/checkout.css 931 B
release/woocommerce-payments/dist/checkout.js 33.4 kB
release/woocommerce-payments/dist/express-checkout-rtl.css 229 B
release/woocommerce-payments/dist/express-checkout.css 229 B
release/woocommerce-payments/dist/express-checkout.js 15.1 kB
release/woocommerce-payments/dist/frontend-tracks.js 854 B
release/woocommerce-payments/dist/index-rtl.css 52.6 kB
release/woocommerce-payments/dist/index.css 52.6 kB
release/woocommerce-payments/dist/index.js 302 kB
release/woocommerce-payments/dist/multi-currency-analytics.js 1.08 kB
release/woocommerce-payments/dist/multi-currency-rtl.css 4.46 kB
release/woocommerce-payments/dist/multi-currency-switcher-block.js 60.6 kB
release/woocommerce-payments/dist/multi-currency.css 4.46 kB
release/woocommerce-payments/dist/multi-currency.js 57.3 kB
release/woocommerce-payments/dist/order-rtl.css 730 B
release/woocommerce-payments/dist/order.css 730 B
release/woocommerce-payments/dist/order.js 42 kB
release/woocommerce-payments/dist/payment-gateways-rtl.css 1.32 kB
release/woocommerce-payments/dist/payment-gateways.css 1.32 kB
release/woocommerce-payments/dist/payment-gateways.js 38.4 kB
release/woocommerce-payments/dist/plugins-page-rtl.css 386 B
release/woocommerce-payments/dist/plugins-page.css 386 B
release/woocommerce-payments/dist/plugins-page.js 20.1 kB
release/woocommerce-payments/dist/product-details-rtl.css 433 B
release/woocommerce-payments/dist/product-details.css 436 B
release/woocommerce-payments/dist/product-details.js 12.3 kB
release/woocommerce-payments/dist/settings-rtl.css 11.6 kB
release/woocommerce-payments/dist/settings.css 11.5 kB
release/woocommerce-payments/dist/settings.js 224 kB
release/woocommerce-payments/dist/subscription-edit-page.js 703 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal-rtl.css 524 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.css 524 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.js 20.2 kB
release/woocommerce-payments/dist/subscription-product-onboarding-toast.js 730 B
release/woocommerce-payments/dist/subscriptions-empty-state-rtl.css 120 B
release/woocommerce-payments/dist/subscriptions-empty-state.css 120 B
release/woocommerce-payments/dist/subscriptions-empty-state.js 19.3 kB
release/woocommerce-payments/dist/tokenized-express-checkout-rtl.css 229 B
release/woocommerce-payments/dist/tokenized-express-checkout.css 229 B
release/woocommerce-payments/dist/tokenized-express-checkout.js 15.9 kB
release/woocommerce-payments/dist/tos-rtl.css 235 B
release/woocommerce-payments/dist/tos.css 235 B
release/woocommerce-payments/dist/tos.js 21.8 kB
release/woocommerce-payments/dist/woopay-direct-checkout.js 6.13 kB
release/woocommerce-payments/dist/woopay-express-button.js 24.8 kB
release/woocommerce-payments/dist/woopay-rtl.css 4.31 kB
release/woocommerce-payments/dist/woopay.css 4.28 kB
release/woocommerce-payments/dist/woopay.js 71 kB
release/woocommerce-payments/includes/subscriptions/assets/css/plugin-page.css 625 B
release/woocommerce-payments/includes/subscriptions/assets/js/plugin-page.js 814 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/i18n-loader.js 2.46 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/jetpack-script-data.js 767 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/i18n-loader.js 1.02 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/script-data.js 69 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/babel.config.js 163 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.css 2.47 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.js 14.2 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.rtl.css 2.47 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.css 10 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.js 28.4 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.rtl.css 10 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.css 198 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.js 280 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.rtl.css 198 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.css 625 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.js 333 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.rtl.css 626 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-users.js 424 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-ajax.js 521 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-callables.js 585 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-admin-create-user.css 215 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-admin-create-user.js 521 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-login.css 721 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-login.js 412 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-users.js 632 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/about.css 1.04 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-empty-state.css 294 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-order-statuses.css 408 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin.css 3.59 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/checkout.css 301 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/modal.css 746 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/view-subscription.css 574 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/wcs-upgrade.css 414 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin-pointers.js 543 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin.js 9.4 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.js 6.78 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.min.js 3.84 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-coupon.js 545 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-subscription.js 2.52 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.js 22.2 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.min.js 11.7 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/payment-method-restrictions.js 1.29 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/wcs-meta-boxes-order.js 507 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/payment-methods.js 358 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/single-product.js 428 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/view-subscription.js 1.38 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/wcs-cart.js 782 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/modal.js 1.09 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/wcs-upgrade.js 1.26 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.css 391 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.js 3.04 kB

compressed-size-action

includes/class-logger.php Show resolved Hide resolved
src/Internal/Logger.php Outdated Show resolved Hide resolved
src/Internal/Logger.php Outdated Show resolved Hide resolved
src/Internal/Logger.php Outdated Show resolved Hide resolved
src/Internal/LoggerContext.php Outdated Show resolved Hide resolved
src/Internal/LoggerContext.php Outdated Show resolved Hide resolved
$entry_context = is_array( $context ) && array_key_exists( 'context', $context )
? $context['context']
: [];
if ( ! array_key_exists( 'source', $entry_context ) || 'woopayments' !== $entry_context['source'] ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use Logger::LOG_FILENAME instead of woopayments here:

Suggested change
if ( ! array_key_exists( 'source', $entry_context ) || 'woopayments' !== $entry_context['source'] ) {
if ( ! array_key_exists( 'source', $entry_context ) || Logger::LOG_FILENAME !== $entry_context['source'] ) {

Overall, I was about to suggest significantly simplifying this set of conditions, but I realized that you might want to use the $entry_context to add to the log. Is that the case? Because otherwise, the third parameter for Logger->log() is useless, as it does not automatically appear in the output.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we can eventually drop the changes to pass context to the Logger, as it won't make sense for WooPayments. On an interesting note, WooCommerce logger appends context information, if it is present, to the end of the log entry.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WooCommerce logger appends context information, if it is present, to the end of the log entry.

I tested this briefly, and the context that I provided did not appear in the log file. I'll look into how Woo core does it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not in the WooPayments log, because entries are processed by the new filter. But if you disable filter, you will see it.

@dmvrtx dmvrtx self-assigned this Dec 5, 2024
@dmvrtx dmvrtx marked this pull request as ready for review December 5, 2024 11:12
Copy link
Contributor

@RadoslavGeorgiev RadoslavGeorgiev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI I tinkered with a small parser, you can find it here: gist. You should be able to add it to the plugin codebase, and alter the log path.

The PR contains plenty of meaningful changes, and IMO is heading in the correct direction: We can identify failing requests, and there is plenty of context to understand what's going on.

The biggest topic for me is grouping the logs for requests. The PoC parser does not have any filters at the moment, but when it does, ex. by API URL or response status, a lot of relevant entries would be hidden.

includes/class-logger-context.php Outdated Show resolved Hide resolved
Comment on lines +35 to +36
public static function set_value( $key, $value ) {
wcpay_get_container()->get( InternalLoggerContext::class )->set_value( $key, $value );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing to do with this PR in particular, but when we started the re-engineering project, and added src with the DI container, I was really hoping that we'd be able to refactor the whole plugin to work that way. Thus this feels like a hack, but I understand it is necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't aware about it, but sounds interesting. I think we way src is made to work is much easier to follow than the long lists of include or require operators in the other parts of plugin code.

Comment on lines 2111 to 2115
Logger::log( "REQUEST $method $redacted_url" );
Logger::log(
'HEADERS: '
. var_export( $headers, true ) // phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_var_export
);
Logger::log( Logger::format_object( 'HEADERS', $headers ) );

if ( null !== $body ) {
Logger::log(
'BODY: '
. var_export( $redacted_params, true ) // phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_var_export
);
Logger::log( Logger::format_object( 'BODY', $redacted_params ) );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that we have the context, do we need to log things here as separate entries?

I imagine that we could store everything in context, and then only call log() once, either for the error, or successful response.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it can be done in the next optimisation. I see the point, that we can log one Request object, but I'd do that if we use a similar object internally. However I think we can follow the structure of remote_request arguments when logging.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d16d5c5

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After implementing the change I've noticed that we log redacted version of the arguments. I will update it, to follow the same structure, but use redacted data.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in c47ea0c

*/
public static function format_object( $label, $object ) {
try {
$encoded = wp_json_encode( $object, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_THROW_ON_ERROR );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small suggestion here, but could we add a string like JSON: at the beginning here? Then the parser would not be required to guess whether it is JSON that should be parsed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 13933bb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enhanhce logged data to help with debugging support requests
3 participants